An Analysis of Logistic Models: Exponential Family Connections and Online Performance

Logistic models are arguably one of the most widely used data analysis techniques. In this paper, we present analyses focussing on two important aspects of logistic models—its relationship with exponential family based generative models, and its performance in online and potentially adversarial settings. In particular, we present two new theoretical results on logistic models focusing on the above two aspects. First, we establish an exact connection between logistic models and exponential family based generative models, resolving a long-standing ambiguity over their relationship. Second, we show that online Bayesian logistic models are competitive to the best batch models, even in potentially adversarial settings. Further, we discuss relevant connections of our analysis to the literature on integral transforms, and also present a new optimality result for Bayesian models. The analysis makes a strong case for using logistic models and partly explains the success of such models for a wide range of practical problems.